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DETAILED ACTION 

Information Disclosure Statement 

1 . The information disclosure statement (IDS) submitted on 8/1 8/2003 was 
considered by the examiner. 

Specification 

2. The disclosure is objected to because of the following informalities: paragraph 
[19] has the "image data region" labeled "114" and the "audio data region" labeled "112", 
but in the fig. 1 the "image data region" is labeled "112" and the "audio data region" is 
labeled "1 14". In paragraph [29] the "personal computer is labeled "120", but in fig. 2 
the "personal computer" is labeled "202". In paragraph [34] the "memory element" is 
labeled "112" three times in the paragraph, but in fig. 1 the "memory element" is labeled 
"106". All of the above numbers should be changed in the specification to match the 
numbers in the drawings also listed above. In paragraph [37] applicant refers to 3-D 
audio clip 312, the 3-D should be changed to 2-D. 

Appropriate correction is required. 

Claim Objections 

3. The numbering of claims is not in accordance with 37 CFR 1-126 which requires 
the original numbering of the claims to be preserved throughout the prosecution. When 
claims are canceled, the remaining claims must not be renumbered. When new claims 
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are presented, they must be numbered consecutively beginning with the number next 
following the highest numbered claims previously presented (whether entered or not). 

Misnumbered claims 12-21 will be examined according to their given 
dependencies but they must be renumbered so that the claims are 1-20 rather than 1- 
10 and then 12-21. 

4. Claim 4 is objected to because of the following informalities: claim 4 is 
dependent on claim 4. For the purposes of examination claim 4 will be read as 
dependant upon claim 3. Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

6. Claims 1-2, 6-10, 12, and 16-21 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Hu (US 2002/0158129). 

Consider claim 1, Hu explicitly teaches "a digital camera (display 
apparatus), comprising: 

a photosensor (image sensor 58, fig 2B) to capture an image of a bar code 
comprising audio information (to attain full-image view of an encoded data 54, 
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paragraph [0017], lines 13-16, where encoded data 54 can be a two-dimensional 
high-density bar code, paragraph [01 1 1], lines 16-19); 

a processor (processor 72, fig 9) to determine audio information from the 
captured image of the bar code (coordinates the overall task of scanning, 
decoding and playing back the audio data, paragraph [01 12], lines 4-6); and 

a speaker (loudspeaker 38) to generate audible sound corresponding to 
the determined audio information (for sound reproduction, paragraph [0116], 
lines)." 

Consider claim 2, Hu explicitly teaches "the digital camera of claim 1 , 
further comprising a memory wherein logic resides, the logic for determining the 
audio information from the captured image of the bar code when the logic is 
executed by the processor (a read-only memory (ROM) 77 stores the machine 
code routines for execution by processor 72, such as the algorithm for decoding 
encoded data 54, paragraph [0113], lines 14-16)." 

Consider claim 6, Hu explicitly teaches "a method for retrieving audio 
information with a digital camera (display apparatus), the method comprising the 
steps of: 

capturing an image of a bar code with the digital camera (image sensor 
attains a full-image view of an encoded data 54, paragraph [0107], lines 13-16 
and encoded data 54 can be a two-dimensional high-density bar code, paragraph 
[0111], lines 16-19); 
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determining audio information corresponding to the captured image of the 
bar code with the digital camera (the processor 72 decodes encoded data 54, 
paragraph [01 14], lines 19-20, where the encoded data is the bar code of audio 
information); and 

generating an audible sound corresponding to the determined audio 
information with an audio reproduction device residing in the digital camera (the 
sound analog signal goes to the audio amplifier 78 for amplification and then onto 
the loudspeaker 38 for sound reproduction, paragraph [0116], lines 5-7)." 

Consider claim 7, Hu explicitly teaches "the method of claim 6, further 
comprising the step of communicating the determined audio information to the 
audio reproduction device such that an audible sound corresponding to the 
determined audio information is generated by the audio reproduction device (the 
sound analog signal goes to the audio amplifier 78 for amplification and then onto 
the loudspeaker 38 for sound reproduction, paragraph [0116], lines 5-7, and fig 9 
shows that the information from the processor 72 goes to the digital signal 
processor 80 and then finally through the amplifier 78 and the loudspeaker 38)." 

Consider claim 8, Hu explicitly teaches that "the audio reproduction device 
is an electro-audio transducer (loudspeaker 38)." 

Consider claim 9, Hu explicitly teaches that "the audio reproduction device 
is a speaker (loudspeaker 38)." 
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Consider claim 10, Hu explicitly teaches "determining audio information 
further comprises the step of executing logic with a processor residing in the 
digital camera (information in the ROM 77 is used with processor 72), the logic 
for determining the audio information from the captured image of the bar code 
when the logic is executed by the processor (a read-only memory (ROM) 77 is 
used to store the machine code routines for execution by processor 72, such as 
the algorithm for decoding encoded data 54, paragraph [01 13], lines 14-16)." 

Consider claim 12, Hu explicitly teaches "the step of retrieving the logic 
from a memory residing in the digital camera (a read-only memory (ROM) 77, 
which is inside the camera, is used to store the machine code routines and send 
them to the processor for execution by processor 72, such as the algorithm for 
decoding encoded data 54, paragraph [0113], lines 14-16)." 

Consider claim 16, Hu explicitly teaches "the step of capturing the image 
of the bar code further comprises capturing the image of the bar code selected 
from a group consisting of a one dimensional (l-D) bar code, a two dimensional 
(2-D) bar code and a three dimensional (3-D) bar code (two-dimensional high- 
density bar code formats may also be utilized, paragraph [01 1 1], lines 16-17)." 

Consider claim 17, Hu explicitly teaches "a system for retrieving audio 
information with a digital camera (display apparatus), comprising: 

means for capturing an image of a bar code with the digital camera (image 
sensor 58, fig 2B is used to attain full-image view of an encoded data 54, 
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paragraph [0017], lines 13-16, where encoded data 54 can be a two-dimensional 
high-density barcode, paragraph [0111], lines 16-19); 

means for processing the captured image of the bar code into digital data 
(processor 72, fig 9, coordinates the overall task of scanning, decoding and 
playing back the audio data, paragraph [0112], lines 4-6 and sends information to 
the digital signal processor 80); 

means for determining audio information from the digital data (processor 
72, fig 9, coordinates the overall task of scanning, decoding and playing back the 
audio data, paragraph [0112], lines 4-6 and sends information to the digital signal 
processor 80); and 

means for generating an audible sound corresponding to the determined 
audio information (loudspeaker 38 for sound reproduction, paragraph [01 16], 
lines)." 

Consider claim 18, Hu explicitly teaches "the system of claim 17, wherein 
the means for determining audio information further comprises means for 
executing logic, the means for executing residing in the digital camera, and the 
logic for determining the audio information from the captured image of the bar 
code when the logic is executed (a read-only memory (ROM) 77 stores the 
machine code routines for execution by processor 72, such as the algorithm for 
decoding encoded data 54, paragraph [01 13], lines 14-16)." 
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Consider claim 19, Hu explicitly teaches "a computer-readable medium 
having a program for retrieving audio information with a digital camera, the 
program comprising logic to perform the steps of: 

determining audio information from a captured image of a printed bar code 
(a read-only memory (ROM) 77 stores the machine code routines for execution 
by processor 72, such as the algorithm for decoding encoded data 54, paragraph 
[0113], lines 14-16); 

communicating the determined audio information to a speaker such that an 
audible sound corresponding to the determined audio information is generated by 
the speaker (a read-only memory (ROM) 77 stores the machine code routines for 
execution by processor 72, such as the algorithm for decoding encoded data 54, 
paragraph [01 13], lines 14-16 once the data is processed by the processor it is 
sent to the digital signal processor 80 and then to the audio amplifier 78 and 
loudspeaker 38, fig 9)." 

Consider claim 20, Hu explicitly teaches "the computer-readable medium 
of claim 1 9, further comprising logic for: 

receiving captured image data corresponding to the bar code from a 
photosensor (image sensor 58 sends information to the processor 72 which is 
run by the ROM as explained above, fig 9 and image sensor 58, fig 2B Is used to 
attain full-image view of an encoded data 54, paragraph [0017], lines 13-16, 
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where encoded data 54 can be a two-dimensional high-density bar code, 
paragraph [0111], lines 16-19) residing in the digital camera; 

generating an audio signal corresponding to the audio information (the 
ROM controls the processor which decodes the coded data 54 and then sends 
that information to the digital signal processor 80 and processor 72, fig 9, 
coordinates the overall task of scanning, decoding and playing back the audio 
data, paragraph [0112], lines 4-6); and 

communicating the audio signal to the speaker (the information is then 
sent to the audio amplifier 78 and then the loudspeaker 38)." 

Consider claim 21, Hu explicitly teaches "the computer-readable medium 
of claim 19, further comprising logic for capturing an image of the printed bar 
code (system only operates when processor is powered on and then the ROM 
controls the processor so the image sensor 58, fig 2B can be used to attain full- 
image view of an encoded data 54, paragraph [0017], lines 13-16, where 
encoded data 54 can be a two-dimensional high-density bar code, paragraph 
[0111], lines 16-19)." 

Claim Rejections - 35 USC § 103 
7. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
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the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

8. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

9. Claims 3-5 and 13-14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hu (US 2002/0158129) in view of Horii et al. (US 7,024,109). 

Consider claim 3, Hu explicitly teaches the a display apparatus that reads bar 
codes from a picture and reproduces the coded sound using a speaker. 

However, Hu does not explicitly teach the recording of a sound, which is 
encoded to a bar code and then printed on the image. 

In the same field of endeavor, Horii et al teach an image processing 
apparatus, camera, with an image sensor for capturing images and relating those 
images to captured sounds. Horii et al. further teach the use of a microphone 
102 "for inputting voice of a user or a person or the like as a subject" (column 10, 
lines 39-40), a CPU 120, which "executes processing relating to inputting of a 
sound" (column 11, lines 17-19), and the idea that the stored sound information 
is read out from the ASIC 103 as code data (column 1 1 , lines 37-40), where the 
code data is a two-dimensional barcode (column 13, lines 31-32). 
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Therefore, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to include the recording of sound, which is 
encoded into a bar code and then printed on the image found in Horrii et al. into 
the display apparatus of Hu, in order to have the capability of capturing sound, at 
any time, and assigning that sound to an image via a bar code, "which stores 
information of multi dimensions and is easy to access" on the printed image 
(column 13, lines 41-43). 

Consider claim 4, Horii et al. further teach the digital camera "further 
comprising a memory wherein logic resides (ASIC 103), the logic for determining 
the bar code corresponding to the detected audible sound when the logic is 
executed by the processor (the sound is stored in the CF card through the ASIC 
103 which provides the coded information to the printer, column 1 1 , lines 28-30, 
fig 12)." It would have been obvious to include the ASIC for the same reasons as 
above. 

Consider claim 5, Horii et al. further teach "means for communicating the 
bar code to a means for printing (the sound stored in the CF card is similarly read 
out to be transferred to the printing section so as to be printed in the form of the 
code data, column 1 1 , lines 37-40), such that the bar code is printed (fig 12 
shows the camera portion connected to the printer section by interface 210, and 
fig 15 shows the printer image with the code data below it)." It would have been 
obvious to include the connection to the printer for the same reasons as above. 
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Consider claim 13, Hu explicitly teaches the a method where a display apparatus 
reads bar codes from a picture and reproduces the coded sound using a speaker. 

However, Hu does not explicitly teach a method of recording a sound, 
which is encoded to a bar code, and then printing the image. 

In the same field of endeavor, Horii et al teach a method of using an image 
processing apparatus, camera, with an image sensor for capturing images and 
relating those images to captured sounds. Horii et al. further teach the use of a 
microphone 102 "for inputting voice of a user or a person or the like as a subject" 
(column 10, lines 39-40), a CPU 120, which "executes processing relating to 
inputting of a sound" (column 11, lines 17-19), and the idea that the stored sound 
information is read out from the ASIC 103 as code data (column 1 1 , lines 37-40), 
where the code data is a two-dimensional bar code (column 1 3, lines 31-32). 

Therefore, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to include the recording of sound, which is 
encoded into a bar code and then printed on the image found in Horrii et al. into 
the method of using the display apparatus of Hu, in order to have the capability of 
capturing sound, at any time, and assigning that sound to an image via a bar 
code, "which stores information of multi dimensions and is easy to access" on the 
printed image (column 13, lines 41-43). 

Consider claim 14, Horii et al. further teach "the steps of: communicating 
the new bar code to the printer (the sound stored in the CF card is similarly read 
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out to be transferred to the printing section so as to be printed in the form of the 
code data, column 1 1 , lines 37-40); and printing the new bar code (fig 12 shows 
the camera portion connected to the printer section by interface 210, and fig 15 
shows the printed image with the code data below it)." It would have been 
obvious to include the step of communicating the bar code to the printer and 
printing the bar code for the same reasons as above. 

10. Claim 15 is rejected under 35 U.S.C. 103(a) as being unpatentable over Hu (US 
2002/0158129) in view of Horii et al. (US 7,024,109) as applied to claims 13 and 14 
above, and further in view of Kamishima et al. (US 6,415,108). 

Consider claim 15, the combination of Hu and Horii et al. as explained above 
explicitly teaches a method of recording sound encoding it into a bar code and then 
printing on the same sheet as the image. 

However, the combination does not explicitly teach the printing of a caption 
along with the bar code data. 

In the same field of endeavor, Kamishima et al. teach a camera with the 
capability of printing a bar code, which corresponds to audio information, on an 
image captured by the camera. Kamishima et al. further teach the printing of a 
caption with information that relates the image data with the recorded sound data 
(column 4, lines 57-67 and column 5, lines 1-7). 
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Therefore, it would have been obvious to one of ordinary skill in the art at 
the time of the invention to include the caption printed on the image in 
Kamishima et al. onto the printed image described in the combination of Hu and 
Hari et al. in order to easily relate the voice code image and the photographed 
still image (column 4, lines 57-58). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Scott Egan whose telephone number is (571) 270-1452. 
The examiner can normally be reached on Monday-Friday 8:00 a.m. - 5:00 p.m., EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Patrick Edouard can be reached on (571) 270-1455. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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